The relationship between patient experience and real-world digital health access in primary care: A population-based cross-sectional study

Implementing digital health technologies in primary care is anticipated to improve patient experience. We examined the relationships between patient experience and digital health access in primary care settings in Ontario, Canada. We conducted a retrospective cross-sectional study using patient responses to the Health Care Experience Survey linked to health and administrative data between April 2019—February 2020. We measured patient experience by summarizing HCES questions. We used multivariable logistic regression stratified by the number of primary care visits to investigate associations between patient experience with digital health access and moderating variables. Our cohort included 2,692 Ontario adults, of which 63.0% accessed telehealth, 2.6% viewed medical records online, and 3.6% booked appointments online. Although patients reported overwhelmingly positive experiences, we found no consistent relationship with digital health access. Online appointment booking access was associated with lower odds of poor experience for patients with three or more primary care visits in the past 12 months (adjusted odds ratio 0.16, 95% CI 0.02–0.56). Younger age, tight financial circumstances, English as a second language, and knowing their primary care provider for fewer years had greater odds of poor patient experience. In 2019/2020, we found limited uptake of digital health in primary care and no clear association between real-world digital health adoption and patient experience in Ontario. Our findings provide an essential context for ensuing rapid shifts in digital health adoption during the COVID-19 pandemic, serving as a baseline to reexamine subsequent improvements in patient experience.

There is a growing need to examine whether the experimental evidence for digital health translates to the real-world context of individuals with wide-ranging health and digital literacies using various technologies.Past evaluations of digital health tools were efficacy-based and did not provide evidence for the everyday use of technologies or diverse groups of users [24][25][26].For example, a cohort study of Ontario primary care patients and providers demonstrated that virtual care improved access and patient experience [24].However, the intervention was limited to synchronous and asynchronous virtual care [24].
Our objective was to examine if pre-pandemic access to telehealth, electronic health records (EHRs), and online appointment booking are associated with the primary care experience of Ontario adults.Our findings describe the influence of digital health and predictors on patient experience and act as a baseline for repeated studies examining rapid digital health adoption and patient experience beyond the pandemic.We hypothesize that digital health access is associated with favourable primary care experiences.

Study design and participants
We conducted a cross-sectional retrospective study to evaluate the association between primary care experience and digital health access in Ontarians �16 years of age.We derived the cohort from the Health Care Experience Survey (HCES).The HCES is a cross-sectional survey used by the Ontario Ministries of Health and Long-Term Care and researchers to measure access and experience across the healthcare system [27][28][29][30].The Institute for Social Research (ISR) at York University conducts the HCES.The sampling frame of the HCES uses the Registered Persons Database (RPDB) to randomly select households of Ontario adults aged 16 years or older.A notification letter is mailed to households selected for the survey, followed by a phone interview (via cellphone or landline) [31].Participants may refuse to answer questions on the HCES and may choose to stop the interview at any point.ISR also collects informed consent from participants over the phone interview to link HCES responses with administrative databases within ICES [30].The response rate for included waves ranged from 29% to 39% [27].
We accessed all de-identified HCES responses linked to administrative datasets through ICES (formally Institute for Clinical Evaluative Sciences) collected between October 28, 2012, and February 20, 2020.ICES fully anonymized all data before being accessed by investigators.
data providers (e.g., healthcare organizations and government) prohibit ICES from making the dataset publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at https://www.ices.on.ca/DAS/AHRQ(email: das@ices.on.ca).The full dataset creation plan and underlying analytic code have been included as supplemental information files.
Our analyses included participants who responded to the HCES from the wave 27 pilot, which introduced digital health questions, to wave 29, the latest data at the time of analysis, as the survey was paused due to the COVID-19 pandemic (April 1, 2019 -February 20, 2020).Participants agreed to have their responses linked with ICES data holdings.We excluded adults who were not linked to or did not self-report having a primary care provider or having a primary care visit within 12 months of the survey date, as several HCES questions are framed within 12 months (Fig 1).

Dependent measures
Like prior studies, our patient experience measure was a linear combination of Likert responses (1 to 5, never to always) to HCES questions on patient experience (Table 1) [32,33].We removed responses: It depends [on] who they see/what they are there for, Not Applicable, Don't know, and Refused from the analysis.

Digital health access
The HCES includes information on access to telehealth, EHRs, and online appointment booking.We classified telehealth access by seven grouped HCES questions regarding primary care access over the past 12 months through telephone, email, video, electronic messaging, or other virtual means.We determined EHR access using three HCES questions that assessed whether participants had viewed medical records using an online system or digital tool over the past 12 months, including systems designed for people with specific health conditions or comprehensive health records.We identified online appointment booking access using one HCES question to which participants responded whether they emailed or visited a website to set up a primary care appointment over the past 12 months.

Covariables
We included variables in the analysis based on hypothesized associations with patient experience, past literature, and frameworks such as Levesque's conceptual framework for healthcare access [33,34].These variables included self-reported financial situation, primary language spoken at home, educational attainment, health status, and years of knowing their primary care provider from the HCES.The RPDB provided the age and sex of participants.We obtained the Rurality Index of Ontario (RIO) from the RPDB and coded it into groups of large urban (RIO: 0), urban (RIO: 1-9), small urban (RIO: 10-39), and rural (RIO: � 40) [35].We obtained Aggregated Diagnoses Groups (ADG) through the Johns Hopkins ACG1 System Version 10.We identified primary care providers rostered or virtually rostered to patients by the highest payment of primary care services over the past 12 months through the Client Agency Program Enrolment (CAPE) and the Ontario Health Insurance Plan Claims (OHIP) databases.We grouped patient enrolment models provided by the Corporate Provider Database (CPDB) and the ICES Physician Database (IPDB) into the following categories: Enhanced Fee-for-Service, Capitation, and Other.OHIP provided the number of primary care visits with the most responsible provider 12 months before the survey date.The Ontario Marginalization Index (ONMARG) provided quintiles for dissemination-area concentrations of material deprivation, dependency, ethnic concentration, and residential instability.S1 Appendix describes the coding of independent variables.

Statistical methods
We compared the characteristics of respondents with missing data to those with complete data to assess bias (systematic vs random missingness) and implemented row-wise deletion for missing covariates.Per communication from the Ministry of Health and the ISR, we coded missing responses to digital health questions as "no access" in line with skip patterns.

Experience questions Coding
When you see your provider or someone else in their office, how often do they know important information about your medical history?
When you see your provider or someone else in their office, how often do they give you an opportunity to ask questions about recommended treatment?
When you see your provider or someone else in their office, how often do they spend enough time with you?
When you see your provider or someone else in their office, how often do they involve you as much as you want to be in decisions about your care and treatment?
When you see your provider or someone else in their office, how often do they explain things in a way that is easy to understand? https://doi.org/10.1371/journal.pone.0299005.t001 We fit an unstratified multivariable logistic regression model with patient experience as the outcome.We included potential confounders through a hierarchical approach, first implementing digital health access, patient demographics and health, healthcare-related factors, and geographic factors [33,36,37].We tested the independence assumption through clustering by providers and LHIN.We fit additional models stratified by the median number of visits.
We conducted a sub-group analysis using multivariable logistic regression to measure how digital health factors were associated with participants reporting "sometimes," "rarely," or "never" to individual HCES experience items, unstratified and stratified by the median age and number of encounters.We removed influential outliers from all models and observed multicollinearity through generalized variance inflation factors (GVIF).We assessed goodness of fit using the c-statistic.A c-statistic larger than 0.70 indicates a good model fit.All analyses were conducted using R Studio (version 1.1.456).

Ethics approval
The Hamilton Integrated Research Ethics Board exempted this study from a formal ethics review as this research project falls under section 45 of Ontario's Personal Health Information Protection Act, which does not require a research ethics board review.The use of the data in this study is authorized and approved by ICES' Privacy and Legal Office, which does not require participant consent according to regulations in Ontario, Canada.

Main results
Summed patient experience.We found no significant association between patient experience and telehealth access (p = 0.45), digital health record access (p = 1.00), and online booking access (p = 0.46).Tables 3 and 4 provide odds ratios for unstratified and stratified models by the number of encounters, respectively.Random effects were not considered in the final model, as the mixed-effect models did not improve fit, and each provider was only responsible for one to two participants.Generalized variance inflation factors did not suggest multicollinearity.The unstratified model had a c-statistic of 0.65, which remained consistent after stratification (� 3 encounters: 0.61; > three encounters: 0.66).
After adjusting, telehealth (aOR 1.00, 95% CI 0.77-1.28),digital health record (aOR 0.97, 95% CI 0.41-2.10),and online appointment booking access (aOR 1.07, 95% CI 0.57-1.93)were not associated with poor experience in participants with three or fewer encounters.In those with over three visits in the past 12 months, participants with online booking access had an 84% reduction in the odds of reporting poor primary care experience than those with no online booking access over the past 12 months (aOR 0.16, 95% CI 0.02-0.56).However, telehealth access (aOR 0.83, 95% CI 0.59-1.17)and digital health record access (aOR 0.82, 95% CI 0.31-1.92)were nonsignificant in the same subgroup.Other factors, including younger age, tight financial circumstances, primary languages other than English, and fewer years with a dedicated primary care provider, were associated with less favourable experiences.Financially and ethnically marginalized communities reported poorer experiences, consistent with financial comfort and language barriers (Tables 3  and 4).
Sub-group analysis.Participants who accessed health records electronically had a 65% decrease in the odds of reporting their primary care provider sometimes, rarely, or never being aware of details of their medical history compared to those without access (aOR 0.35, 95% CI 0.11-0.88).In addition, participants with greater than three encounters over the past 12 months who used telehealth had a 37% decrease in odds of reporting their primary care provider sometimes, rarely, or never providing the opportunity to ask questions compared to those without access (aOR 0.63, 95% CI 0.41-0.96).Older adults over 60 who accessed telehealth also had a 33% decrease in the odds of reporting their provider sometimes, rarely, or never spent enough time with them compared to the same reference group (aOR 0.67, 95% CI 0.46-0.98).We observed other associations among experience items and confounders (S3 Appendix).

Interpretation
We found no consistent association between access to online appointment booking and digital health records and primary care patient experience.However, our sub-group analysis suggested telehealth access was associated with enhanced communication and time spent with 1 Defined as summed first quintile of patient experience scores (� 21) 2 Includes telephone, email, video, electronic messaging, or other virtual means of communication with primary care provider 12 months before the survey 3 Includes digital medical record access 12 months before the survey, including systems specific to health conditions or comprehensive records 4 Defined by the Ontario Marginalization Index as the proportion of the population without an income generated from employment, including ratios of seniors and unemployment within the population 5 Defined by the Ontario Marginalization Index as a measure of poverty and access to basic human needs 6 Defined by the Ontario Marginalization Index as the concentration of recent immigrants and visible minorities 7 Defined by the Ontario Marginalization Index as the level of family and housing instability 8 Rurality Index of Ontario primary care providers.Patient experience is likely influenced more heavily by other provider factors rather than digital health tools as facilitators of patient-centeredness. Contrary to our findings, past research has observed that patient access to digital health records improves patient-provider communication and overall experiences [14,38].A need for greater adoption to measure any potential effect may explain these mixed outcomes.Digital health technologies are enablers rather than solutions, and providers need payment 2 Includes telephone, email, video, electronic messaging, or other virtual means of communication with primary care provider 12 months before the survey 3 Includes digital medical record access 12 months before the survey, including systems specific to health conditions or comprehensive records 4 John Hopkins Aggregated Diagnosis Groups 5 Enhanced fee for service 6 Rurality Index of Ontario 7 Defined by the Ontario Marginalization Index as the proportion of the population without an income generated from employment, including ratios of seniors and unemployment within the population 8 Defined by the Ontario Marginalization Index as a measure of poverty and access to basic human needs 9 Defined by the Ontario Marginalization Index as the concentration of recent immigrants and visible minorities 10 Defined by the Ontario Marginalization Index as the level of family and housing instability https://doi.org/10.1371/journal.pone.0299005.t003mechanisms to support their adoption.Despite no observed association in the summed score, telehealth access was consistent with experimental pilot studies, where patients who accessed telehealth reported similar or improved experiences and patient-provider communication [24,39,40].Our findings aligned with international population-based studies and literature reviews reporting poorer experience scores among younger, financially, or ethnically marginalized adults [33,36].While we did not observe sex differences in patient experience, similar studies indicate minimal association [33].Our findings were consistent with past studies 2 Includes telephone, email, video, electronic messaging, or other virtual means of communication with primary care provider 12 months before the survey 3 Includes digital medical record access 12 months before the survey, including systems specific to health conditions or comprehensive records 4 John Hopkins Aggregated Diagnosis Groups 5 Enhanced fee for service 6 Rurality Index of Ontario 7 Defined by the Ontario Marginalization Index as the proportion of the population without an income generated from employment, including ratios of seniors and unemployment within the population 8 Defined by the Ontario Marginalization Index as a measure of poverty and access to basic human needs 9 Defined by the Ontario Marginalization Index as the concentration of recent immigrants and visible minorities 10 Defined by the Ontario Marginalization Index as the level of family and housing instability https://doi.org/10.1371/journal.pone.0299005.t004 observing longstanding patient-provider relationships to play a crucial role in shared decisionmaking and patient experience, where longer relationships create greater trust between patients and providers [37].Unlike our study, past studies have observed that participants in worse health have poorer experiences and fewer participatory visits [33,36,37].We measured this association through self-reported health status and ADG values.However, we did not observe a noticeable effect in experience or their involvement in decision-making.Unlike our findings, a past multicenter US study has observed that adults with greater educational attainment were more likely to report shared decision-making with their providers [37].However, the study did not account for language barriers, which played a prominent role in communication and experience with primary care providers.
The proportion of Ontario adults who had a virtual visit dramatically increased to 29.2% in 2021, compared to 1.3% in 2019 [41].While our findings serve as a necessary baseline of experience, access to new digital health and primary care experience data allows us to expand our research beyond the pandemic.System-wide adoption of digital health may introduce health equity and digital literacy concerns.Older adults lacking computer skills or with severe illness and individuals with poor socioeconomic status, cognitive impairments, or disabilities related to language and understanding face additional barriers to accessing technology and digital health [42][43][44][45].In response to the COVID-19 pandemic and Ontario's Digital Health Playbook, primary care digitization must be coupled with evidence and re-evaluated for its impact on patient experience while considering the socioeconomic and digital literacy barriers to patient-centred care and digital health adoption [46].

Limitations
The study uses data from before the COVID-19 pandemic, therefore serving as a baseline for future research examining post-pandemic trends in experience and digital health access.Most study participants were older, while 16.7% of Ontarians are �65 years of age [47].The older demographic may limit the adoption of digital health and bias the sample toward adults who frequently access primary care and have longstanding patient-provider relationships [48,49].The survey is provided to English or French-speaking Ontarians in private residences with a valid address in RPDB, 16 years or older, with a valid health insurance card and active phone number [31].The HCES sampling frame would potentially exclude individuals who do not speak English or French, institutionalized individuals, refugees, persons experiencing homelessness, some Indigenous persons, and persons without access to a phone [28,31].Further, ISR did not survey individuals who were not physically or mentally healthy enough, resulting in an underrepresentation of persons with cognitive impairments and other conditions [28].
Patient experience encompasses all interactions with primary care and is influenced by expectations of care [8].We were unable to measure associations longitudinally or through provider characteristics.Future research should observe long-term associations with experience and implement further health provider and practice factors suggested by frameworks, including the 4 Ps of Patient Experience [50].Our analysis used complete cases of data without any further imputation.The full and analyzed cohorts were generally consistent in demographic characteristics (Table 2).